The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages. This promotes the potential of utilizing models pretrained with data more than 3D as teachers for cross-modal knowledge transferring. In this paper, we revisit masked modeling in a unified fashion of knowledge distillation, and we show that foundational Transformers pretrained with 2D images or natural languages can help self-supervised 3D representation learning through training Autoencoders as Cross-Modal Teachers (ACT). The pretrained Transformers are transferred as cross-modal 3D teachers using discrete variational autoencoding self-supervision, during which the Transformers are frozen with prompt tuning for better knowledge inheritance. The latent features encoded by the 3D teachers are used as the target of masked point modeling, wherein the dark knowledge is distilled to the 3D Transformer students as foundational geometry understanding. Our ACT pretrained 3D learner achieves state-of-the-art generalization capacity across various downstream benchmarks, e.g., 88.21% overall accuracy on ScanObjectNN. Codes will be released at https://github.com/RunpeiDong/ACT.

基于激光雷达的本地化方法是用于大规模导航任务的基本模块，例如最后一英里交付和自动驾驶，并且本地化鲁棒性高度依赖于观点和3D功能提取。我们以前的工作提供了一个观点不变的描述符来处理观点差异；但是，全局描述符在无监督聚类中的信号噪声比率低，从而降低了可区分的特征提取能力。我们开发了SphereVlad ++，这是这项工作中一种引起注意的观点不变的位置识别方法。 SphereVlad ++在每个唯一区域的球形视角上投射点云，并通过全局3D几何分布捕获本地特征及其依赖关系之间的上下文连接。作为回报，全局描述符中的群集元素以本地和全球几何形式为条件，并支持SphereVlad的原始视点不变属性。在实验中，我们评估了SphereVlad ++在匹兹堡市的公共Kitti360数据集和自我生成的数据集上的本地化性能。实验结果表明，SphereVlad ++在小甚至完全逆转的视点差异下优于所有相对最新的3D位置识别方法，并显示0.69％和15.81％的成功检索率，比第二好的检索率更好。低计算要求和高时间效率也有助于其用于低成本机器人的应用。

图像BERT使用掩盖图像建模（MIM）预训练成为应对自我监督的表示学习的一种流行实践。一项开创性的作品将MIM作为一个视觉词汇作为分类任务，将连续的视觉信号用于离散的视觉令牌，并使用预先学习的DVAE将其标记为离散的视觉令牌。尽管有可行的解决方案，但不当离散化仍阻碍了图像预训练的进一步改善。由于图像离散化没有基本真相的答案，因此我们认为，即使可以获得更好的``令牌'''，也不应使用唯一的令牌ID分配蒙面的补丁。在这项工作中，我们引入了改进的BERT风格图像预训练方法，即MC-BEIT，该方法执行MIM代理任务，以放松和精致的多选择培训目标。 Specifically, the multi-choice supervision for the masked image patches is formed by the soft probability vectors of the discrete token ids, which are predicted by the off-the-shelf image ``tokenizer'' and further refined by high-level inter-补丁感知诉诸于观察到类似的补丁应该分享其选择。关于分类，分割和检测任务的广泛实验证明了我们方法的优势，例如，预先培训的VIT-B在Imagenet-1K分类上达到了84.1％的TOP-1微调精度，49.2％AP^B和44.0％对象检测和可可的实例分割的AP^m，在ADE20K语义分段上为50.8％，表现优于竞争性对应物。该代码将在https://github.com/lixiaotong97/mc-beit上找到。

在本文中，我们研究了网络多功能增强学习（MARL）的问题，其中许多代理被部署为部分连接的网络，并且每个代理只与附近的代理交互。网络Marl要求所有代理商以分散的方式作出决定，以优化具有网络之间邻居之间的限制通信的全局目标。受到事实的启发，即\ yexit {分享}在人类合作中发挥关键作用，我们提出了一个分层分散的MARL框架，使代理商能够学会与邻居动态共享奖励，以便鼓励代理商在全球合作客观的。对于每个代理，高级策略了解如何与邻居分析奖励以分解全局目标，而低级策略则会学会优化由邻域的高级策略引起的本地目标。两项政策形成双级优化，交替学习。我们经验证明LTOS在社交困境和网络MARL情景中表明现有的现有方法。

尽管取得了巨大的成功，但深入的学习严重遭受鲁棒性;也就是说，深度神经网络非常容易受到对抗的攻击，即使是最简单的攻击。灵感来自脑科学最近的进步，我们提出了一种新的内部模型（DIM），这是一种基于新的生成自动化器的模型来解决这一挑战。模拟人类大脑中的管道进行视觉信号处理，暗淡采用两级方法。在第一阶段，DIM使用丹组器来减少输入的噪声和尺寸，反映了塔马拉姆的信息预处理。从主视觉皮质中的内存相关迹线的稀疏编码启发，第二阶段产生一组内部模型，一个用于每个类别。我们评估了42次对抗攻击的衰弱，表明Dim有效地防御所有攻击，并且优于整体鲁棒性的SOTA。

风险的准确器官（OAR）分割对于减少治疗后并发症的放射治疗至关重要。达人指南推荐头部和颈部（H＆N）区域的一套超过40桨的桨，然而，由于这项任务的可预测的禁止劳动力成本，大多数机构通过划定较小的桨子和忽视的少数，选择了大量简化的协议与其他桨相关的剂量分布。在这项工作中，我们提出了一种使用深度学习的新颖，自动化和高效的分层OAR分段（SOARS）系统，精确地描绘了一套全面的42 H＆N OAR。 SOARS将42桨分层进入锚，中级和小型和硬质子类别，通过神经结构搜索（NAS）原则，专门为每个类别提供神经网络架构。我们在内在机构中使用176名培训患者建立了SOAR模型，并在六个不同的机构中独立评估了1327名外部患者。对于每个机构评估，它始终如一地表现出其他最先进的方法至少3-5％的骰子得分（在其他度量的相对误差减少36％）。更重要的是，广泛的多用户研究明显证明，98％的SOARE预测只需要非常轻微或没有直接临床验收的修订（节省90％的辐射脑神经工作负载），并且它们的分割和剂量准确度在于或小于帧 - 用户的变化。这些调查结果证实了H＆N癌症放射疗法工作流OAR描绘过程的强烈临床适用性，提高了效率，全面性和质量。

Deep nets generalize well despite having more parameters than the number of training samples. Recent works try to give an explanation using PAC-Bayes and Margin-based analyses, but do not as yet result in sample complexity bounds better than naive parameter counting. The current paper shows generalization bounds that're orders of magnitude better in practice. These rely upon new succinct reparametrizations of the trained net -a compression that is explicit and efficient. These yield generalization bounds via a simple compression-based framework introduced here. Our results also provide some theoretical justification for widespread empirical success in compressing deep nets.Analysis of correctness of our compression relies upon some newly identified "noise stability"properties of trained deep nets, which are also experimentally verified. The study of these properties and resulting generalization bounds are also extended to convolutional nets, which had eluded earlier attempts on proving generalization.

As natural language processing (NLP) for gender bias becomes a significant interdisciplinary topic, the prevalent data-driven techniques such as large-scale language models suffer from data inadequacy and biased corpus, especially for languages with insufficient resources such as Chinese. To this end, we propose a Chinese cOrpus foR Gender bIas Probing and Mitigation CORGI-PM, which contains 32.9k sentences with high-quality labels derived by following an annotation scheme specifically developed for gender bias in the Chinese context. Moreover, we address three challenges for automatic textual gender bias mitigation, which requires the models to detect, classify, and mitigate textual gender bias. We also conduct experiments with state-of-the-art language models to provide baselines. To our best knowledge, CORGI-PM is the first sentence-level Chinese corpus for gender bias probing and mitigation.

We present Second Thought, a new learning paradigm that enables language models (LMs) to re-align with human values. By modeling the chain-of-edits between value-unaligned and value-aligned text, with LM fine-tuning and additional refinement through reinforcement learning, Second Thought not only achieves superior performance in three value alignment benchmark datasets but also shows strong human-value transfer learning ability in few-shot scenarios. The generated editing steps also offer better interpretability and ease for interactive error correction. Extensive human evaluations further confirm its effectiveness.

Three-dimensional (3D) ultrasound imaging technique has been applied for scoliosis assessment, but current assessment method only uses coronal projection image and cannot illustrate the 3D deformity and vertebra rotation. The vertebra detection is essential to reveal 3D spine information, but the detection task is challenging due to complex data and limited annotations. We propose VertMatch, a two-step framework to detect vertebral structures in 3D ultrasound volume by utilizing unlabeled data in semi-supervised manner. The first step is to detect the possible positions of structures on transverse slice globally, and then the local patches are cropped based on detected positions. The second step is to distinguish whether the patches contain real vertebral structures and screen the predicted positions from the first step. VertMatch develops three novel components for semi-supervised learning: for position detection in the first step, (1) anatomical prior is used to screen pseudo labels generated from confidence threshold method; (2) multi-slice consistency is used to utilize more unlabeled data by inputting multiple adjacent slices; (3) for patch identification in the second step, the categories are rebalanced in each batch to solve imbalance problem. Experimental results demonstrate that VertMatch can detect vertebra accurately in ultrasound volume and outperforms state-of-the-art methods. VertMatch is also validated in clinical application on forty ultrasound scans, and it can be a promising approach for 3D assessment of scoliosis.